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Stress  is  a  psychological  construct  with  important  consequences  for  human  health.  A 
substantial  number  of  stress  measures  are  available  that  vary  in  length  and  dimensionality.  The  purpose 
of  this  study  was  to  determine  whether  one  of  these  measures,  the  Perceived  Stress  Questionnaire 
(PSQ),  could  be  shortened  in  a  psychometrically  sound  fashion  to  enhance  its  usability.  The  original 
PSQ  has  30  items,  and,  more  recently,  a  20-item  version  was  proposed.  We  used  existing  data  from  Air 
Force  nursing  personnel  ( n  =  250)  to  shorten  the  PSQ  through  a  combination  of  classical  test  theory 
and  item  response  theory  techniques.  The  resulting  six-item  measure,  the  PSQ6,  had  scale  scores  that 
correlated  highly  with  PSQ30  and  PSQ20  scale  scores  (intraclass  correlation  coefficient  =  .92),  strong 
internal  consistency  (Cronbach’s  a  =  .85),  and  associations  with  stigma,  perceived  barriers,  and 
resilience  scale  scores  that  were  similar  in  strength  to  longer  versions  of  the  PSQ. 

Keywords :  psychological  stress,  psychometrics,  military  personnel 
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Throughout  what  is  now  the  longest  period  of  ongoing  warfare  in  U.S.  history, 
“extraordinary  demands  and  stressors”  (Institute  of  Medicine,  2014,  p.  9)  have  been  placed  on 
the  service  members  of  the  U.S.  military.  The  demand  for  mental  health  services  is  high,  and 
service  members’  beliefs  about  their  ability  to  cope  with  stress  sometimes  affects  their  attitudes 
and  decisions  about  seeking  treatment  (Langston  et  al.,  2010).  Manifestations  of  stress  from  a 
deployment  include  “increased  divorce  rates,  spouse  and  child  abuse,  mental  distress,  substance 
abuse — but  one  of  the  most  troubling  manifestations  is  suicides”  (Ramchand,  Acosta,  Bums, 
Jaycox,  &  Pemin,  2011,  p.  xiii).  Service  members  may  avoid  or  delay  seeking  treatment  out  of 
reluctance  to  admit  they  may  be  having  difficulties  due  to  work-  or  family-related  stress  (Britt  et 
al.,  201 1).  Rigorously  and  pragmatically  assessing  levels  of  occupational  stress  and,  relatedly, 
tailoring  interventions  for  stress  for  military  personnel  are  urgent  issues. 

Measuring  Stress 

The  Perceived  Stress  Questionnaire  (PSQ)  is  a  30-item  instrument  that  was  developed  to 
measure  general  or  recent  stressors  in  clinical  psychosomatic  research  (Levenstein  et  al.,  1993). 
Compared  with  other  stress  scales,  the  PSQ  allows  an  individual  to  express  a  response  to 
stressful  items  that  are  “abstract  enough  to  be  applicable  to  adults  of  any  age,  stage  of  life,  sex, 
or  occupation  [and]  specific  to  a  variety  of  real-life  situations”  (Fliege  et  al.,  2005,  p.  78). 
Although  Fliege  et  al.  (2005)  tested  a  shortened  20-item  version  of  the  PSQ,  a  truly  pragmatic 
version  of  the  PSQ  does  not  currently  exist.  According  to  Glasgow  and  Riley  (2013),  such  a 
pragmatic  stress  measure  would  need  to  have  the  minimum  properties  of  being  important  to 
stakeholders,  low  burden,  actionable,  and  sensitive  to  change.  In  particular,  Glasgow  and  Riley 
recommended  that  a  pragmatic  measure  should  “take  no  longer  than  2  minutes  for  the  typical 
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respondent  to  complete”  (p.  239).  According  to  Fliege  et  al.s  the  20-item  PSQ  takes  slightly  less 
than  5  minutes  to  complete. 

The  general  prompt  for  the  PSQ,  used  in  the  current  study,  asks  respondents  to  rate 
experiences  within  the  last  year  or  two  on  a  4-point  scale  ranging  from  almost  never  to  usually 
(Levenstein  et  al.,  1993).  Details  about  the  PSQ’s  internal  consistency  reliability,  test-retest 
reliability,  convergent  validity,  criterion  validity,  construct  validity,  and  sensitivity  to  change  can 
be  found,  for  example,  in  Levenstein  et  al.  (1993),  Fliege  et  al.  (2005),  and  Kocalevent  et  al. 
(2007)  and  reflect  generally  strong  psychometric  properties.  In  the  study  by  Levenstein  et  al.,  in 
particular,  the  PSQ  scale  score  had  strong  positive  correlations  with  trait  anxiety  (r  =  .69), 
somatic  symptoms  (r  =  .50),  and  Cohen’s  Perceived  Stress  Scale  (r  =  .56;  Cohen,  Kamarck,  & 
Mermelstein,  1983). 

Stress  in  Military  Nursing  Personnel 

Two  recent  articles  used  the  PSQ  to  measure  perceived  levels  of  stress  experienced  by 
active  component,  Air  Force  registered  nurses  and  medical  technicians  (Author,  2016,  2017). 
Findings  showed  that  44%  of  Air  Force  nursing  personnel  reported  a  stress  or  emotional  problem 
at  the  time  of  the  study  and  that  higher  levels  of  stress  were  associated  with  higher  levels  of 
stigma  about  seeking  mental  health  services  and  lower  resilience  (Author,  2016).  Additionally, 
there  was  a  moderate-strength  positive  association  between  PSQ  scores  and  having  sought 
treatment  in  the  previous  6  months  (Author,  2017).  The  current  study  sought  to  develop  a 
shortened  version  of  the  PSQ  using  the  source  data  for  the  previous  two  articles  (Author,  2016, 
2017).  We  hypothesized  that  this  shortened  stress  instrument  could  reliably  replicate  those 
findings. 


Methods 
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Sampling  and  Measures 

An  anonymous  electronic  survey  was  e-mailed  to  approximately  1,400  active  component 
Air  Force  nursing  personnel  across  three  study  sites  from  October  2014  to  April  2015,  resulting 
in  250  responses  (Author,  2016, 2017).  The  survey  included  30  items  from  the  PSQ  (Levenstein 
et  al.,  1993),  six  Stigma  and  five  Barriers  items  from  scales  present  in  Hoge’s  Perceived  Barriers 
items  (Hoge  et  al.,  2004;  Kim,  Britt,  Klocko,  Riviere,  &  Adler,  2011),  25  items  from  the  Connor- 
Davidson  Resilience  Scale  (Connor  &  Davidson,  2003),  and  a  dichotomous  item  on  treatment- 
seeking  in  the  last  6  months  that  was  asked  of  respondents  who  indicated  that  they  were 
“currently  experiencing  a  stress  or  emotional  problem.”  Details  about  items,  response  options, 
scoring,  reliability,  and  validity  for  these  survey  scales  can  be  found  in  Author  (2016)  along  with 
descriptive  statistics  for  these  scales  and  demographic  characteristics  of  the  sample.  Briefly,  64% 
of  respondents  were  female,  78%  were  White,  9%  were  Hispanic,  56%  were  officers,  and  more 
than  half  (n  =  141;  56%)  had  been  deployed  since  September  11, 2001  (Author,  2016).  Of  the 
1 17  respondents  in  this  sample  who  reported  a  current  stress  or  emotional  problem,  only  about 
one  quarter  (n  =  32;  27%)  reported  seeking  mental  health  treatment  (Author,  2017). 

Data  Analysis 

Stata  14  (Release  14;  StataCorp,  2015)  was  used  for  all  data  analysis  and  a=.05  was  used 
as  the  threshold  for  statistical  significance  for  all  tests.  Values  greater  than  .7  and  .8  for 
Cronbach’s  alpha  were  taken  as  indicating  acceptable  and  good  internal  consistency  reliability, 
respectively. 

The  development  of  a  candidate  short  version  of  the  30-item  PSQ  (PSQ30)  began  by 
fitting  confirmatory  factor  analysis  (CFA)  models  for  each  of  the  four  subscales  of  the  20-item 
version  of  the  PSQ  (PSQ20)  and  for  its  four  subscale  scores  (Fliege  et  al.,  2005)  on  an  overall 
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stress  construct.  Modification  indices  for  the  CFA  model  for  the  overall  stress  construct  were 
used  to  identify  up  to  one  possible  subscale  score  error  covariance  term  to  be  included  in  its 
model  specification.  Indicators  for  “good”  fit  for  all  CFA  models  included  standardized  root 
mean  residual  <  .06,  comparative  fit  index  (CFI)  >  .95,  and  Tucker-Lewis  index  (TLI)  >.95,  with 
TLI  >  .90  considered  “acceptable”  fit. 

Following  the  CFA  models,  item  response  theory  (IRT)  graded  response  models  were 
used  to  identify  relatively  low  information  items  within  subscales  of  the  PSQ20,  and  such  items 
were  removed  in  backward  selection  process,  from  five  items  to  three  items  for  each  subscale.  In 
cases  in  which  the  item  information  function  for  a  single  item  of  the  three  remaining,  backwardly 
selected  items  in  a  subscale  dominated  the  item  information  function  for  the  remaining  two 
items,  that  single  item  was  retained  as  a  candidate  for  the  PSQ  short  version.  In  cases  in  which 
comparably  strong  item  information  functions  for  two  items  dominated  the  item  information 
function  for  a  third  remaining  item,  both  such  items  were  retained  as  candidates  for  the  PSQ 
short  version.  In  this  way,  one  to  two  items  of  each  subscale  of  the  PSQ20  were  retained  as 
candidates  for  the  PSQ  short  version. 

All  candidate  items  for  the  PSQ  short  version  were  fit  on  a  single-factor  CFA  model  that 
included  error  covariance  terms  between  items  retained  from  the  same  PSQ20  subscale  and 
between  items  retained  from  any  PSQ20  subscales,  with  an  error  covariance  term  specified 
between  them  in  the  overall  stress  construct  CFA  model.  Strength  of  associations  among  scale 
scores  for  the  PSQ30,  the  PSQ20,  and  the  candidate  PSQ  short  version  were  assessed  by 
calculating  intraclass  correlation  coefficients  (ICCs).  Pearson’s  correlations  were  used  to 
examine  strength  of  associations  among  the  PSQ30,  PSQ20,  and  candidate  PSQ  short  version 
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scale  scores  and  scale  scores  for  Stigma,  Barriers,  and  Resilience,  as  well  as  the  treatment¬ 
seeking  indicator  variable. 

Results 

The  CFA  and  IRT  analyses  resulted  in  a  six-item  candidate  short  version  of  the  PSQ 
(PSQ6).  As  shown  in  Tables  1  and  2,  CFA  models  for  the  PSQ20  subscales,  the  PSQ20  overall 
stress  construct,  and  the  PSQ6  displayed  acceptable-to-good  fit  with  acceptable-to-good  internal 
consistency.  One  item,  “you  are  lighthearted,”  on  one  subscale  of  the  PSQ20  showed  a  factor 
loading  below  .5.  All  items  on  the  PSQ6  had  factor  loadings  near  or  exceeding  .7,  and  all 
included  terms  in  all  CFA  models  were  statistically  significant. 

Scale  scores  for  the  PSQ30  and  the  PSQ20  were  nearly  perfectly  correlated,  and  the  scale 
score  for  the  PSQ6  was  highly  correlated  with  scale  scores  for  both  the  PSQ30  and  PSQ20 
(Table  3).  Consistent  with  mathematical  expectations  of  the  impact  of  increasing  measurement 
error  with  fewer  items,  Table  4  displays  an  overall  tendency  toward  correlation  attenuation  in 
associations  of  criterion  variables  with,  successively,  the  PSQ30,  PSQ20,  and  PSQ6.  However, 
the  degree  of  attenuation  was  remarkably  slight.  Overall,  the  correlations  between  the  PSQ6  and 
criterion  variables  were  nearly  identical  in  strength,  direction,  and  significance  to  those  observed 
in  earlier  work  using  the  PSQ30  (Author,  2016, 2017). 

Discussion 

Our  analysis  shows  that  a  6-item  candidate  measure,  the  PSQ6,  is  strongly  associated 
with  the  PSQ30  in  this  study  sample  and  retains  similar  psychometric  properties  to  both  the 
PSQ30  and  PSQ20,  albeit  at  the  expense  of  granularity.  Regarding  the  use  of  the  PSQ6  as  a 
pragmatic  measure  for  studies  of  stress  involving  military  personnel,  our  analysis  is  preliminary, 
but  there  are  indications  of  promise.  We  lack  data  regarding  response  burden  and  importance  to 
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stakeholders  of  the  PSQ6,  but  presumably  completion  time  for  a  six-item  reduced  inventory 
would  be  far  less  than  for  the  parent  questionnaire  or  the  20-item  alternative.  Given  the  high 
correlation  (ICC  =  .92)  between  the  PSQ6  and  PSQ30,  it  seems  reasonable  to  expect  that  the 
PSQ6  would  share  the  demonstrated  sensitivity  to  change  properties  of  the  PSQ30  (Fliege  et  al., 
2005);  however,  that  remains  to  be  tested  in  future  research.  The  extent  to  which  PSQ6  scores 
are  actionable,  Glasgow  and  Riley’s  (2013)  other  minimum  criterion  for  a  pragmatic  measure,  is 
currently  unclear.  Further  research  would  be  needed  to  elucidate  how  the  PSQ6  could  be 
incorporated  into  screening  or  tailored,  participatory,  evidence-based  interventions. 

Important  limitations  of  this  study  include  a  low  survey  response  rate  in  the  original  study 
and  concerns  about  the  generalizability  of  a  study  focused  on  Air  Force  nursing  personnel  to 
other  military  personnel.  Independent  verification  of  the  psychometric  properties  of  the  PSQ6 
and  its  relationship  to  the  PSQ30  in  new  or  existing  study  samples  would  strengthen  the  results 
of  this  study,  as  would  analyses  of  relationships  between  the  PSQ6  and  PSS4,  the  four-item 
version  of  Cohen’s  Perceived  Stress  Scale  (Cohen  et  al.,  1983).  Although  the  PSQ6  shows 
promise  as  a  reliable  measure  of  overall  stress,  either  of  the  longer  versions  of  the  PSQ  would  be 
more  appropriate  where  finer-grained,  dimensional  analysis  of  stress  is  wanted. 

Conclusion 

This  study  is  responsive  to  the  importance  of  measuring  stress  in  military  personnel  and 
could  serve  as  a  step  toward  more  pragmatically  assessing  interventions  that  aim  to  ameliorate 
such  stress.  Based  on  our  findings,  the  PSQ6  is  a  potential  low-burden  instrument  that  could  be 
used  to  measure  stress  being  experienced  by  military  personnel.  Future  research  is  currently  in 
the  planning  stages  to  assess  the  psychometric  properties  of  the  PSQ6  in  a  new  sample  of 


SHORTENED  STRESS  MEASURE  WITH  NURSING  PERSONNEL 


8 


military  nursing  personnel  and,  potentially,  its  usefulness  in  providing  actionable  information  to 
health  care  providers  serving  broader  military  populations. 
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Table  1 


Factor  Loadings  and  Fit  for  the  PSQ20  Scale  and  Subscales  (N  =  239) 

Items  and  Subscales 

Coef  (SE) 

Alpha, b  CFI, 

TLI,  SRMR 

Worries 

.85, .96, .93, .03 

You  fear  you  may  not  manage  to  attain  your  goals 

.74  (.04) 

You  feel  frustrated 

.73  (.04) 

Your  problems  seem  to  be  piling  up 

.85  (.03) 

You  have  many  worries 

.84  (.03) 

You  are  afraid  for  the  future 

.73  (.04) 

Tension 

.82,  .96,  .92,  .04 

You  feel  rested0 

.73  (.04) 

You  feel  calm0 

.82  (.03) 

You  feel  tense 

.81  (.03) 

You  feel  mentally  exhausted 

.65  (.04) 

You  have  trouble  relaxing 

.70  (.04) 

Joy 

.73,  .99,  .99,  .03 

You  feel  you're  doing  things  you  really  like0 

.68  (.04) 

You  are  full  of  energy0 

.68  (.04) 

You  feel  safe  and  protected0 

.59  (.05) 

You  enjoy  yourself 

.90  (.03) 

You  are  lighthearted0 

.40  (.06) 

Demands 


.82,  .98,  .95,  .03 
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You  feel  that  too  many  demands  are  being  made  on 


you  .80  (.03) 

You  have  too  many  things  to  do  .95  (.02) 

You  feel  you're  in  a  hurry  .66  (.04) 

You  have  enough  time  for  yourself5  .73  (.04) 

You  feel  under  pressure  from  deadlines  .58  (.05) 

PSQ20d 

Worries  .85  (.02) 

Tension  .90  (.02) 

Joyc  .74  (.03) 

Demands  .70  (.04) 

Error  covariance,  Joy  and  Demands®  -.21  (.08) 


.87, 1.00,  .99,  .01 


Note.  Italicized  items  retained  in  PSQ6.  PSQ  =  Perceived  Stress  Questionnaire;  Coef = 
coefficient;  SE  =  standard  error;  CFI  =  comparative  fit  index;  SRMR  =  standardized  root  mean 
residual;  TLI  =  Tucker-Lewis  index. 

“Standardized  regression  coefficient.  bCronbach's  alpha.  ®Reverse-coded  item  or  subscale  score. 
dModel  fit  using  subscale  scores.  Gp  =  .006;  p  <  .001  for  all  other  coefficients. 
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Table  2 

Factor  Loadings  and  Fit  for  the  PSQ6  Scale  (N  =  239) 


Items 

Coef  (SE) 

Alpha, b  c  CFI,  TLI,  SRMR 

Your  problems  seem  to  be  piling  up 

.76  (.04) 

.85, .98, .94,  .03 

You  have  many  worries 

.72  (.04) 

Error  covariance,  problems  and  worries 

.40  (.07) 

You  feel  mentally  exhausted 

.80  (.03) 

You  have  trouble  relaxing 

.76  (.04) 

Error  covariance,  exhausted  and  trouble 

.27  (,08)d 

You  enjoy  yourself® 

.68  (.05) 

You  have  too  many  things  to  do 

.73  (.04) 

Error  covariance,  enjoy  and  to  do 

-.37  (.09) 

Note.  PSQ  =  Perceived  Stress  Questionnaire;  Coef.  =  coefficient;  CFI  =  comparative  fit  index; 
SRMR  =  standardized  root  mean  residual;  TLI  =  Tucker-Lewis  index. 

Standardized  regression  coefficient.  bCronbach's  alpha.  ca  =  .95  for  the  PSQ30;  a  =  .92  for  the 
PSQ20.  dp  =  .001;  p  <  .001  for  all  other  coefficients.  eReverse-coded  item. 
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Table  3 

Correlations  Among  the  PSQ30,  PSQ20,  and  PSQ6  Scale  Scores  (N  =  239) 


PSQ30 

PSQ20 

Scale 

ICC  (95%  Cl) 

ICC  (95%  Cl) 

PSQ30 

PSQ20 

.98  [.97,  .99] 

PSQ6 

.92  [.90,  .94] 

.92  [.88,  .95] 

Note,  p  <  .001  for  all  ICC  estimates.  PSQ  =  Perceived  Stress  Questionnaire;  ICC  =  intraclass 
correlation  coefficient;  Cl  =  confidence  interval. 
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Table  4 

Convergent  Validity  of  PSQ  Scale  Scores  (N  =  239) 


Scores 

PSQ30 

r(SE) 

PSQ20 

r(SE) 

PSQ6 

r(SE) 

Stigma 

.41  (.05) 

.38  (.05) 

.36  (.05) 

Barriers 

.23  (.06) 

.22  (.06) 

.18a  (.06) 

Resilience 

-.53  (.04) 

-.54  (.04) 

-.48  (.05) 

Treatment-seekingb 

.30  (.06) 

.29  (.06) 

.31  (.06) 

Note.  PSQ  =  Perceived  Stress  Questionnaire.  SE  =  standard  error. 

ap  =  .003;  p  <  .001  for  all  other  values  of  Pearson's  r.  hn  =  234. 

